PAC Learning Using Nadaraya-Watson Estimator Based on Orthonormal Systems
نویسندگان
چکیده
Regression or function classes of Euclidean type with compact support and certain smoothness properties are shown to be PAC learnable by the Nadaraya-Watson estimator based on complete orthonormal systems. While requiring more smoothness properties than typical PAC formulations, this estimator is computationally efficient, easy to implement, and known to perform well in a number of practical applications. The sample sizes necessary for PAC learning of regressions or functions under sup norm cost are derived for a general orthonormal system. The result covers the widely used estimators based on Haar wavelets, trignometric functions, and Daubechies wavelets.
منابع مشابه
Weighted Nadaraya-Watson Regression Estimation
In this article we study nonparametric estimation of regression function by using the weighted Nadaraya-Watson approach. We establish the asymptotic normality and weak consistency of the resulting estimator for-mixing time series at both boundary and interior points, and we show that the estimator preserves the bias, variance, and more importantly, automatic good boundary behavior properties of...
متن کاملOn the Adaptive Nadaraya-watson Kernel Regression Estimators
Nonparametric kernel estimators are widely used in many research areas of statistics. An important nonparametric kernel estimator of a regression function is the Nadaraya-Watson kernel regression estimator which is often obtained by using a fixed bandwidth. However, the adaptive kernel estimators with varying bandwidths are specially used to estimate density of the long-tailed and multi-mod dis...
متن کاملNadaraya-Watson estimator for sensor fusion problems
The classical Nadaraya-Watson estimator is shown to solve a generic sensor fusion problem where the underlying sensor error densities are not known but a sample is available. By employing Haar kernels this estimator is shown to yield finite sample guarantees and also to be efficiently computable. TWO simulation examples, and a robotics example involving the detection of a door using arrays of u...
متن کاملLarge and moderate deviations principles for kernel estimators of the multivariate regression
Abstract : In this paper, we prove large deviations principle for the Nadaraya-Watson estimator and for the semi-recursive kernel estimator of the regression in the multidimensional case. Under suitable conditions, we show that the rate function is a good rate function. We thus generalize the results already obtained in the unidimensional case for the Nadaraya-Watson estimator. Moreover, we giv...
متن کاملSequential Fixed-width Confidence Bands for Kernel Regression Estimation
We consider a random design model based on independent and identically distributed (iid) pairs of observations (Xi, Yi), where the regression function m(x) is given by m(x) = E(Yi|Xi = x) with one independent variable. In a nonparametric setting the aim is to produce a reasonable approximation to the unknown function m(x) when we have no precise information about the form of the true density, f...
متن کامل